Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate Space Agon Demo from using Open Match 1 to Open Match 2 #51

Merged
merged 7 commits into from
Nov 5, 2024
Merged

Migrate Space Agon Demo from using Open Match 1 to Open Match 2 #51

merged 7 commits into from
Nov 5, 2024

Conversation

peterzhongyi
Copy link
Contributor

@peterzhongyi peterzhongyi commented Sep 17, 2024 •
edited
Loading

This pull request changes the Space Agon's match making logic from using Open Match 1 to Open Match 2(https://github.com/googleforgames/open-match2/blob/main)

Key files to review:

  1. director/director.go
  2. frontend/frontend.go
  3. mmf/mmf.go
  4. install/helm/space-agon/templates && install/helm/space-agon/values.yaml
  5. omclient/omclient.go - mostly copied from https://github.com/googleforgames/open-match-ecosystem/blob/v2.0/v2/internal/omclient/omclient.go, but added an initializer and assignment related APIs.
  6. logging/logging.go - completely copied from https://github.com/googleforgames/open-match-ecosystem/blob/v2.0/v2/internal/logging/logging.go.

Key differences between OM1 and OM2:

  1. Frontend and Director communicates with OM1 via gRPC, but use HTTP to communicate with OM2. Thus the omclient.go and logging.go was created.
  2. OM1 was previously deployed in Kubernetes via helm, but OM2 is now deployed on Cloud Run (https://github.com/googleforgames/open-match2/blob/main/docs/DEVELOPMENT.md).
  3. Redis was previously deployed in Kubernetes via helm, but is now deployed on Memorystore on Pantheon UI.
  4. MMF is now an internal load balancer service because OM2 is not on Kubernetes.

Two tests are intentionally left out in the pull request:

  1. Frontend/frontend_test.go was deleted because the OM2 repo doesn't provide a omtest tool anymore, and the test itself isn't originally covering a lot of the code logic either.
  2. The integration test was skipped in the PR checks because the process of deploying the Space Agon demo has changed a lot, and since OM2 and Redis are no longer deployed on Kubernetes, it is more complicated to set up for the integration test to run. Also, once it's all set up, it's more reasonable to test the demo just by running it, rather than running the integration test.

To run the demo with this PR, first follow the README to until you installed Agones. Then instead of installing Open Match with make openmatch-install, follow the steps below to install OM2:

  1. Find the region of your Kubernetes Cluster, and use this region for the steps below.
  2. Create a Redis instance in Memorystore on Pantheon UI.
  3. Deploy OM2 by cloning the https://github.com/googleforgames/open-match2/blob/main repo and following the docs/DEVELOPMENT.md file to install OM2. You need to replace the IP address of the OM_REDIS_WRITE_HOST and OM_REDIS_READ_HOST in the service.yaml file to be the IP address of the Redis instance you created.

Now to allow your frontend to talk to the OM2 you installed:

  1. Create a Google Service Account in your project and give it the Cloud Run Invoker role
  2. Replace the value of iam.gke.io/gcp-service-account field in frontend.yaml to be the Google Service Account you created.
  3. Bind the Kubernetes Service Account in frontend.yaml to the Google Service Account(in the form of SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com) with
gcloud iam service-accounts add-iam-policy-binding   --role roles/iam.workloadIdentityUser   --member "serviceAccount:PROJECT_ID.svc.id.goog[default/frontend-sa]"   YOUR_GOOGLE_SERVICE_ACCOUNT
  1. In install/helm/space-agon/values.yaml, change the omCoreAddress field of frontend to be the address of you OM2 service on Cloud Run.

Because the MMF server are now called from Cloud Run instead of Kubernetes Cluster, the MMF needs to have an IP that the director needs to know when the director is starting up. To do this,

  1. Reserve an internal IP address for you project in Pantheon UI.
  2. In install/helm/space-agon/values.yaml, replace the ipAddress field for mmf and the mmfAddress field for director to be the IP address that you just reserved.
  3. In install/helm/space-agon/values.yaml, change the omCoreAddress field of director to be the address of you OM2 service on Cloud Run.

Now in the root directory of the demo and use make build && make install to start the Space Agon demo. NOTE: after you have the external IP address of the frontend service, open two chrome windows so you can see them AT THE SAME TIME, and click Find Game. If you can't see both tabs at the same time, the demo's client has a bug that will connect to a match several times.

Towards #50

@peterzhongyi peterzhongyi force-pushed the exp branch 2 times, most recently from ab76c7d to cb2c632 Compare September 17, 2024 21:10
@kemurayama kemurayama self-requested a review September 19, 2024 00:38
kemurayama
kemurayama reviewed Sep 20, 2024
View reviewed changes
install/helm/space-agon/values.yaml Outdated
servicePort: 50502
image:
repository: local/space-agon-mmf
tag: TAG
imagePullPolicy: IfNotPresent
imagePullPolicy: Always
ipAddress: 10.128.0.30
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we deploy the space-agon on public cloud such as GKE, it is difficult to specify the ip address in advance.

Is this IP address used in the Open Match 2 ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed on GKE, by default the IP address is random. However, you can reserve an IP address for your GKE cluster and use that reserved IP address for the service. By doing so, the service will always be turned up with that specified IP address. I outlined the reasons and steps in the PR description as well.

Copy link
Contributor Author

@peterzhongyi peterzhongyi Sep 20, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correction: it's not "reserve an IP address for your GKE cluster", it's "reserve an IP address in your vpc network" by going into "VPC Network" page on Pantheon UI, then "IP addresses" tab and then "RESERVE INTERNAL STATIC IP ADDRESS"

@kemurayama
Copy link
Collaborator

Sorry for being late. I'll check the changes.

@kemurayama
Copy link
Collaborator

@peterzhongyi

Thank you for this excellent PR! I've deployed it locally and verified it worked with Cloud Run.

Open Match 2 seems to have increased dependencies on Google Cloud, such as Cloud Run, Memorystore for Redis, and Cloud IAM configuration.

To enable integration tests on GitHub Actions and make the deployment simple, I'm considering deploying Open Match 2 Core and Redis on Kubernetes.

If possible, could you merge the Kubernetes deployment version of Open Match 2 as the main code change and document the Cloud Run deployment as an optional method in such as DEPLOYMENT.md ?

Since your current PR steps works as is, simply documenting these steps would be sufficient.

@peterzhongyi
Copy link
Contributor Author

peterzhongyi commented Oct 28, 2024 •
edited
Loading

@kemurayama Thanks so much for going through the steps, and I'm glad it worked!

And yes, currently Open Match 2 indeed has a much stronger reliance on Google Cloud. This is a decision made by people including @joeholley in an effort to justify Open Match's product value to Google. I'm clearly aware that this change would mean less portability between different Clouds, which is one of the original goals of such an open sourced project.

In fact, I've already had some discussions with my team, and have come to the conclusion that providing direct support for Kubernetes deployment is still vital to OM2. As a result, I'm planning to add the Kubernetes deployment version to in a coming PR. I decided to do this in a different PR because there will be some significant work involved to support Kubernetes.

However, since the original design of OM2 was mostly around deploying through Cloud Run, I think it's still a good idea to merge this PR to the om2 branch, and I can update the README.me for this branch as well to document the deployment strategy. Meanwhile, I can start working on the other PR to support Kubernetes. As for the integration tests, I think we'll have to skip them for now, until I come up with the next PR.

What do you think?

@kemurayama
Copy link
Collaborator

Hi @peterzhongyi,

Thanks !! I'm glad to know that OM2 also support Kubernetes in coming PRs. I'll merge this request.

@kemurayama kemurayama merged commit b11c2e6 into googleforgames:om2 Nov 5, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kemurayama kemurayama kemurayama left review comments

@govargo govargo govargo left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@peterzhongyi @kemurayama @govargo